Evaluating the Performance of Public Urban Transportation Systems in India

Evaluating the performance of public transportation systems facilitates operational improvement and strategic decisions. The purpose of this study was to evaluate the relative performance of 26 public urban transportation organizations in India using various criteria. We grouped these 19 criteria as Operations, Finance, and Accident-based. First, we evaluated the importance of these criteria groups using the Analytic Hierarchy Process (AHP). Then, we evaluated the organizations (Decision Making Units, DMUs) using various criteria within each criteria group using Data Envelopment Analysis (DEA). Finally, a Transportation Efficiency Number (TEN) was developed that quantified the overall performance of the DMUs considering the 19 criteria. Included is a discussion on the applicability of this approach, thus helping practicing managers understand the lacuna, if any, and set mutual benchmarks and benefits from the experience of others. This approach helps make strategic decisions for policy-making and achieves better results. Introduction In India, more than 30 percent of the population lives in an urban area. Road public transport and railways are the commonly-used modes of local transportation. Hand-pulled, cycle , and auto rickshaws, taxis, and hired two-wheelers (in the state of Goa) are examples of privately-operated road public transport. Buses and specially-designed Bus Rapid Transit Systems (BRTS) are shared modes of local transportation. Various government bodies (such as the state government or municipal corporations) manage the shared mode of road transportation. In this paper, the shared mode of transportation is called a road public transportation system. Literature Review Evaluating the performance of a road public transport system is essential for making suitable amendments in its improvement strategy. Various studies have been carried out for evaluating performance in this area. For instance, Cruz et al. (2012) evaluated the performance of urban transportation in Portugal. To evaluate the efficiency of 52 small and 43 big cities, Data Envelopment Analysis (DEA) was used. The authors used four different Evaluating the Performance of Public Urban Transportation Systems in India Journal of Public Transportation, Vol. 17, No. 4, 2014 175 DEA-based efficiency-benchmarking models. Holmgren (2012) conducted a stochastic frontier analysis-based study to evaluate the efficiency of public transport systems in Sweden. This analysis was based on the data collected from 1986 to 2009 to illustrate the change in efficiency over time. The reasons for development in the region were the emphasis on highly-dense routes and effective implementation of environmental and safety standards. Yu and Fan (2009) applied a Mixed Structure Network Data Envelopment Analysis (MSNDEA) model to evaluate the performance of multimodal bus transit in Taiwan. This model represents a consumption process and was used to estimate the production efficiency, service effectiveness, and operational effectiveness of multimodal transit firms. To study the logistics strategy implemented in Guatemala and the United States, an empirical study was conducted by McGinnis et al. (2012). It appeared that the logistics managers in Guatemala were more inclined towards marketing and information strategies rather than process-driven strategies. To measure the service quality in urban bus transport, Barabino et al. (2012) applied a modified SERVQUAL model. The main purpose of the study was to develop an evaluation tool to verify the service quality standard offered. Based on the data collected during a two-week survey, various attributes were confirmed, including on-board security, bus reliability, cleanliness, and bus frequency. With a view toward understanding the areas of improvement of public transportation in Dublin, Kinsella and Caulifield (2011) conducted a survey, the results of which reflected that visitors or newcomers to a city are less concerned with the traditional aspects of public transport service quality and, instead, are more concerned with information and reliability. Another survey was conducted by Sullivan (1984), which presented some interesting observations about the performance of public surface transportation in the U.S. and Canada. The summary presents the expected developments in the economy and comments on various land use trends. Lin (2010) developed a framework to evaluate the performance of stochastic transportation systems. The research focused on measuring the quality level of a transportation system. The author proposed a performance index to identify the probability of the upper bound of system capacity that equals a demand vector subject to budget constraints. This algorithm, based on minimal cuts, generated maximal capacity to meet demand exactly, given the budget. Then, the performance index was evaluated. Mishra et al. (2012) studied the performance indicators for public transit connectivity in multimodal transportation networks. The objective of this work was to quantify and evaluate transit services in terms of locations for funding, providing service delivery strategies, and assessing the efficiency and effectiveness. The authors illustrated their approach with an example and a network in the region of Washington–Baltimore and claimed to offer reliable indicators as a tool for determining connectivity of the multimodal transportation network. For evaluating the performance in railway, Yu and Lin (2008) proposed a DEA-based framework to estimate passenger and freight technical efficiency and service effectiveness. The authors selected 20 railways for the study and suggested various strategies for improving operational performance. Some studies that cite exclusive application of DEA (or extensions of DEA) in the area of transportation-related decision-making are briefly presented. Hawas et al. (2012) applied Evaluating the Performance of Public Urban Transportation Systems in India Journal of Public Transportation, Vol. 17, No. 4, 2014 176 DEA to evaluate the performance of Al Ain public bus service. The evaluation enabled an investigation of the chances of reducing operating costs given the prevailing conditions. The presented approach also helped to demonstrate improvement in performance by minor modifications in the route alignment. Hahn et al. (2013) applied a network-based DEA approach to evaluate the performance of bus companies in Seoul, Korea. The authors simultaneously used both desirable and undesirable output parameters. Several policy decisions made based on this study were the expansion of bus transit systems, additional bus stops, reduction of taxes etc. Sanchez (2009) presented a comparative analysis of public bus transport in Spain. DEA, principal component analysis, and Tobit regression were used for this analysis. The authors showed that efficiency levels are not related to the form of ownership (public vs. private). Another finding of this study indicated six percent surplus resources. Barnum et al. (2007) developed a performance indicator (efficiency score) using DEA and Stochastic Frontier Analysis and illustrated its application to the park-and-ride lots of the Chicago Transit Authority. The authors demonstrated the suitability of the approach from a transit agency perspective to identify sub-unit inefficiencies and claim the usefulness of approach for improving both sub-unit and system performance. Suzuki and Nijkamp (2011) presented an approach by integrating the Distance Friction Minimization model, Context-Dependent model, and Charnes, Cooper, and Rhodes (CCR) DEA methodology. This approach developed a stepwise efficiency-improving projection for conventional DEA. The authors presented an application of the proposed approach for public transport operations in Japan. Liu et al. (2013) presented a literature review on the applications of DEA. This research indicated wide application of DEA in the area of banking, health care, agriculture and farm, transportation, and education. A key feature of this paper was the development of trajectory in each application area through main path analysis. The authors also suggested that two-step contextual analysis and network DEA are the recent trends across applications. Some works apply DEA for analyzing the performance of support systems of transportation systems. In analyzing a downtown space reservation system considering various perspectives (such as service provider, user, and the community), Zhao et al. (2011) presented two DEA-based models, radial and slacks-based. The results showed that the analysis could lead to improved designs of a downtown space reservation system. For analyzing environmental efficiency in a Chinese transportation system, Chang et al. (2013) presented a non-radial DEA model with the slacks-based measure. The results indicated that the environmental efficiency levels in most of the provinces is lower than 50 percent of the target level. While there is sufficient existing literature to evaluate performance considering different parameters and/or a single criterion, there seems to be a need to conduct a performance study based on various criteria. In the present work, we looked at 19 criteria to evaluate the performance of the public road transportation system in India. We grouped these criteria under three categories—Operations, Finance, and Accident-based. We evaluated 26 state and/or municipal transportation systems (Decision Making Units, DMUs). Using the CCR model of DEA, we evaluated the performance of the DMUs in each category. Evaluating the Performance of Public Urban Transportation Systems in India Journal of Public Transportation, Vol. 17, No. 4, 2014 177 This resulted in a performance number by assigning weights (importance) to the criteria groups using AHP. The analysis carried out considered the data compiled over the fiscal year ending March 2011. This paper is organized as follows. In the next section, we briefly explain the DEA and AHP approaches. Then, we discuss the approach for the performance evaluation of various transportation systems in India. Finally, we present discussion and conclusions. Appendix 1 provides a list of DMUs, and Appendix 2 shows the various criteria within each criteria group used for evaluation. Performance Analysis Tools Data Envelopment Analysis (DEA) DEA is a well-known non-parametric benchmarking tool based on linear programming. Farrell (1957) initially developed the concept of DEA, and later, Charnes, Cooper, and Rhodes (1978) developed this approach. The CCR model measures the relative efficiency of a set of firms (DMUs) that use a variety of inputs to produce a range of outputs under the assumption of constant return to scale (CRS). In DEA, the aim is to measure the performance of a DMU using the concept of efficiency or productivity, defined as the ratio of total weighted outputs to total weighted inputs. While measuring the performance, this model captures the productivity inefficiency of a firm based on its actual scale size and its inefficiency based on its actual scale (Banker 1984). The best performing unit in the set of DMUs is assigned a score of 100 percent (1), and the remaining DMUs are assigned a score ranging between 0 and 100 percent (0 and 1) relative to the score of best-performing DMU. DEA forms a linear efficiency frontier that passes through the best-performing units within the group, and all remaining less-efficient units lie off the frontier. The term “efficiency” used in DEA is relative efficiency. The DEA formulation for mth DMU under consideration is as follows: Where, ηm is the efficiency of mth DMU Yjm is the jth output of the mth DMU Vjm is the weight of jth output Evaluating the Performance of Public Urban Transportation Systems in India Journal of Public Transportation, Vol. 17, No. 4, 2014 178 Xim is the ith input of the mth DMU Uim is the weight of ith input Yjn and Xin are the jth output and ith input of the nth DMU Analytic Hierarchy Process (AHP) Saaty (1980) initially proposed the AHP, a multi-criteria decision-making tool. AHP has a wide range of applications (Vaidya and Kumar 2006) and involves following steps: 1. Problem decomposition and hierarchy construction: Construct the overall hierarchical structure; identify the criteria. 2. Determination of alternatives: Identify the decision alternatives. 3. Pairwise comparison: Determining the relative importance of the identified criteria; the decision-maker needs to provide a score as the preferences for each pair in the hierarchy. 4. Weight calculation and consistency check: Calculate priority weights for each level using a mathematical normalization method. A consistency ratio also is calculated. The value of a consistency ratio greater than 10 percent indicates that the decisionmaker is not consistent. A review of scores is essential in such cases. In case of group decision-making, a geometric mean of scores is considered. 5. Hierarchy synthesis: Integrate the priority weights at different hierarchical levels to allow overall evaluation of alternatives, leading to a decision-making strategy. (In the present study, we conducted a single-hierarchy AHP. Therefore, this step may not be essential.) Proposed Framework In this section, we explain the proposed three-phase framework. Initially, using AHP, the weights of the criteria groups were determined. Then, to compute the efficiency within each criteria group, DEA was used. Finally, we computed the Transportation Efficiency Number (TEN) to reflect the overall performance of the DMUs. Phase 1 Initially, using the AHP approach, we assigned weights to each criteria group in terms of their importance. Group decision-making involving various stakeholders such as commuters, employees, practicing managers, and members of the governing body can be useful in such situations. These values are called Criteria Importance Value (CIV). The CIV for Operations, Finance, and Accident-based group criteria were designated as (CIV)o, (CIV)f, and (CIV)a, respectively. Phase 2 Within each criteria group, for each DMU, we computed efficiency using the CCR DEA approach. The efficiencies computed for the Operations, Finance, and Accident-based criteria groups were designated as ηio , ηif , and ηia , respectively, where i is the DMU. It should be noted that the criteria within the Operations and Finance criteria groups follow a higher (output-input ratio) is better principle, i.e., benefit criteria. However, the criteria Evaluating the Performance of Public Urban Transportation Systems in India Journal of Public Transportation, Vol. 17, No. 4, 2014 179 classified under the Accident-based group are cost attributes (lower is better). This can be considered an undesirable output. To accommodate this view, we computed the efficiencies by considering the [TRβ] approach presented by Ali and Seiford (1990). Here, a large, scalar β is added to each of the undesirable output values such that the transformed values are positive. The transformation is done using the following expression: fr (Q) = -qr + βr (1) Where r is the output and j is the DMU. Phase 3 For each of the DMUs, we computed TEN as the product of the efficiency and the CIV: (TEN)dmu = (ηo (CIV)o ) + (ηf (CIV)f ) + (ηa (CIV)a ) (2) Analysis In the first phase of the analysis, we assigned weights to criteria groups using a group decision-making approach. A team of three—a commuter, an employee, and a practicing manager—rated the criteria using AHP. A pairwise comparison matrix was determined after considering the geometric mean of the scores of each member. The weights assigned were 0.297, 0.167, and 0.54, respectively, for the Operations, Finance, and Accident-based criteria. Consistency ratios of the scores obtained were within limits. In the next phase of the analysis, we computed efficiency for each DMU within each criteria group, as indicated in Phase 2 of the proposed framework. The data required for this study were a compilation from a report by the Ministry of Road Transport and Highways, Government of India (2011) (see Table 1). DMUs for this study were various state governing bodies or cities, as shown in Appendix 1. As indicated earlier, the input and output criteria were drawn from the Operations, Finance, and Accident-based groups. Appendix 2 provides brief information about the criteria selected for the analysis. Evaluating the Performance of Public Urban Transportation Systems in India Journal of Public Transportation, Vol. 17, No. 4, 2014 180 TABLE 1. Data for Year Ending March 2011 NA NFA AFH SS REK TR RE RPB TC CDT OC SC AFU SP VP FE PKO PKP PC Ahmedabad 538 17 942 5274 525.1 10890.58 2074.2 3167.43 24809.47 4725.16 7215.62 12119.92 674 27.28 152.71 3.47 31700.2 21021 844.7 Andhrapradesh 2879 1047 21802 120566 28958 521485.87 1800.84 6553.2 548366.97 1893.66 6891 236927.87 21701 65.8 363.9 5.17 1462379 973944 582.9 B.E.S.T. 847 49 4652 30183 2615.2 111278.17 4255.1 6553.56 149416.42 5713.45 8799.66 78982.73 4082 23.74 154.02 2.91 176102 123071 904.1 Bangalore 556 88 6110 32953 4580.2 132934.51 2902.37 5960.79 127899.53 2792.44 5735.02 45986.74 5641 38.08 205.38 4.01 223844.2 197604.2 699.6 Calcutta 130 9 956 6102 348.6 6541.41 1876.59 1874.65 25142.74 7212.9 7205.46 17769.53 501 15.65 99.9 3.37 20173 12108 483.2 Chandigarh 125 8 471 2136 439.5 11148.4 2536.78 6484.83 14905.84 3391.78 8670.47 6484.96 444 56.37 255.63 4.09 21974 20215.6 458.6 Delhi 209 62 5771 35557 2920.7 96454.13 3302.43 4579.07 325108.12 11131.17 15434.19 95946.7 4330 22.5 138.66 4.24 197602.3 138010.9 525.4 Gujarat 1010 204 7692 40670 9485.1 196804.31 2074.89 7009.75 212854.15 2244.1 7581.41 85273.6 6327 63.9 337.84 5.53 472465.7 325906.6 286.8 Haryana 296 106 3249 16536 3797.1 85971 2264.13 7249.52 113704 2994.51 9588.11 53523 3079 62.91 320.19 4.78 189854 134796.3 352.7 Karnataka 1278 233 7160 34019 8707.7 207868.28 2387.19 7953.94 201663.03 2315.92 7716.5 63281.65 6574 70.13 333.19 4.85 452798.8 329637.6 324.3 Kolhapur 177 7 135 666 108.4 3188.43 2942.17 6470.68 3423.3 3158.9 6947.34 1373.17 125 44.58 219.93 3.58 4412.1 3019.1 719.8 Maharashtra 3407 445 16214 103565 18973.3 493901 2603.14 8345.59 488878 2576.67 8260.71 194912 15359 50.19 320.6 4.94 879716 543987 428.8 Chennai 1912 133 3414 23540 3471.5 91324.51 2630.67 7328.77 114308.52 3292.74 9173.23 51498.82 3007 40.4 278.59 4.39 24995